CUIML: A language for generating multimodal human-computer interfaces

CUIML, the Cooperative User Interfaces Markup Language, was developed as part of the DWARF project. The goal of DWARF is the development of a framework for augmented reality applications running on wearable computers. For wearable systems, the HCI (human-computer interface) consists of numerous I/O devices such as head mounted displays, palm-size devices, and speech recognition systems. This should lead to a multimodal interaction with the user. To make the development of multimodal user interfaces easier, we propose a unified user interface description language. This language describes the user interface in an abstract way and allows to map it to different output and input components. To keep the different parts of the user interface in a consistent state, a controller object is needed. We used UIML, the User Interface Markup Language, as a starting point. CUIML offers an abstract description language for the views and the controller. By XSL transformations, these presentation free HCI descriptions are converted to markup languages that can be displayed on the various I/O devices. Furthermore the controller, whose internal structure is a deterministic finite automaton (DFA), is also configured by CUIML. CUIML is a promising approach to solving the problems that occur, when developing a multimodal HCI. On a higher level of abstraction CUIML is a description language for views and controllers of the MVC (Model/View/Controller) design pattern that can be transformed to the required structures at runtime. Because it is based on XML it can be extended easily to support many different wearable systems.